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SUBSTITUTE SPECIFICATION 



RELATED APPLICATION 



[0001] This application is related to co-pending patent application no. 10/057,705 entitled, 'The 



Architecture for DSR Client and Server Development Platform", filed January 24, 2002, which 
application is assigned to the assignee of the present application. 

FIELD OF THE INVENTION 

[0002] This application generally relates to distributed speech recognition (DSR), particularly to 
a data transmission system and method for a DSR application. 

BACKGROUND OF THE INVENTION 

[0003] With the growth of the Internet technology and speech recognition technology, both 
speech researchers and computer software engineers have been putting a great deal of effort into 
integrating speech functions with Internet applications. Due to the ease-of-use nature, speech 
recognition technology that provides a convenient input methodology for accessing mobile 
Internet services is becoming more and more important for mobile communication systems. 
[0004] There are alternative architectures, in the art, for speech recognition. The first is a server- 
only processing strategy wherein the speech recognition process is performed only at the server 
side. In this architecture, the client just records the user's voice and transmits the recorded voice 
to the server for processing. The second alternative architecture is a client-only processing 
strategy wherein the recognition process is performed at the client side and only the result of the 
speech recognition is transmitted to the server. The third conventional approach is a client-server 
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processing strategy wherein feature extraction is performed at the client side. Speech feature 
extraction requires only a small part of the computation load needed for the entire procedure of 
speech recognition. The extracted speech features are transmitted from the client to the server 
and then speech recognition is performed at the server side based on the extracted speech 
features. 

[0005] The disadvantage of the first approach is that a high-quality and high-bandwidth 
connection between the client and server is required to support the transmission of voice data. In 
a typical implementation, the recognition performance degrades for data rates below 32 kb/s. The 
second approach has limitations too, because the complexity of medium and large vocabulary 
speech recognition systems are beyond the memory and computational resources of most small 
portable computing devices. The third approach overcomes the disadvantages of the preceding 
two approaches in that less data is transmitted between client and server than the first approach, 
and less computational burden is placed on the client than the second approach. 
[0006] The DSR system, standardized by the European Telecommunications Standards Institute 
(ETSI), is based on the third approach identified above, which overcomes these problems by 
using a low bit rate data channel to send a parameterized representation of the speech from client 
to server, which is suitable for recognition by the server. The speech processing is thus 
distributed between the client terminal and the network. The client terminal performs the speech 
feature parameter extraction, or the front-end processing of the speech recognition system. These 
extracted speech features are transmitted over a data channel to a remote "back-end" recognizer. 
[0007] In spite of the advantages of the conventional DSR application system, the system still 
has particular requirements of data transmission. As the speech features transmitted from DSR 
client to DSR server are packet data not a voice stream, a low bit error rate is required. For the 
interaction (characteristic of conversation) between the DSR server and the DSR client, the 
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typical DSR application system is sensitive to network transmission delay. As a result, the 
typical DSR application system has special Quality of Service (QoS) requirements due to its 
speech-like and data-like characteristics. Moreover, because of the complexity of the network 
between the DSR server, DSR clients and the Web server with which the DSR application 
system operates, data transmission quality, latency, and stability are very important issues in a 
typical DSR application system. 

[0008] Meanwhile, as a packet-oriented extension of the Global System for Mobile 
telecommunication (GSM), well-known GPRS (General Packet Radio Services) can support 
Internet Protocol (IP) protocol and QoS to provide a reliable wireless IP packet transmission 
system with high efficiency. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

[0009] The features of the invention will be more fully understood by reference to the 
accompanying drawings, in which: 

[0010] Figure 1 is an illustrative diagram that shows a DSR application system over a GPRS 
wireless network and the Internet in accordance with an embodiment of the present invention; 
[0011] Figure 2 is a block diagram that depicts an embodiment of a data transmission system for 
a DSR application in accordance with an embodiment of the present invention; 
[0012] Figure 3 is a block diagram that depicts a DSR client wrapper of a data transmission 
system for a DSR application in accordance with an embodiment of the present invention; 
[0013] Figure 4 is a block diagram that depicts a DSR server wrapper of a data transmission 
system for a DSR application in accordance with an embodiment of the present invention; 
[0014] Figure 5 is a flow chart that depicts a method for sending DSR data from a DSR client to 
a DSR server of a DSR application system, in accordance with an embodiment of the present 
invention; 

[0015] Figure 6 is a flow chart that depicts a method for receiving DSR data at a DSR server of a 
DSR application system, in accordance with an embodiment of the present invention. 
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DETAILED DESCRIPTION 

[0016] The structure, operation, advantages, and features of the present invention will become 
apparent in the following detailed description by reference to the accompanying drawings. 
[0017] A DSR application system is an integration of distributed speech recognition and the 
World-Wide Web (WWW). As shown in Figure 1, which is an illustrative diagram that shows a 
DSR application system over a GPRS wireless network and the Internet in accordance with an 
embodiment of the present invention, the DSR application system comprises a plurality of DSR 
clients (101-103), a DSR server (140) and a Web server (150) connecting to the Internet (130). 
There is also a base station (110) and a Gateway GPRS Support Node/Serving GPRS Support 
Node (GGSN/SGSN) (120) between the DSR clients (101-103) and the Internet (130). 
[0018] In this embodiment, the DSR clients (101-103) are mobile terminals of a GPRS wireless 
network, such as mobile phones or other mobile computing devices with GPRS support. As well 
known in the art, GPRS is a packet-oriented extension of GSM, which supports the IP protocol 
and QoS. GGSN and SGSN are used to support wireless/wired interconnection. 
[0019] The DSR application system generally operates in the manner described below: 

1) one of the DSR clients (e.g. DSR client A (101)) first initiates a DSR session with the DSR 
server (140) by sending a request and preference information (such as characteristics of a user's 
speech and voice input device) to the DSR server (140); 

2) upon the receipt of the request, the DSR server (140) sends a DSR Extensible Markup 
Language (DSRML) request to the Web server (150), optionally with the help of a DSR Domain 
Name Service (DNS) (not shown in Figure 1), and the Web server (150) sends back related 
DSRML documents; 

3) after receiving the DSRML documents, the DSR server (140) parses the documents and 
compiles all the grammars that the speech recognition engine needs; 
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4) the DSR server (140) generates display content that is organized as a document comprising 
information cards. DSR server (140) sends the displayable information cards to the DSR client 
(101) and waits for user speech feature extraction data from the DSR client (101); 

5) upon receipt of the displayable information card document, the DSR client (101) displays a 
relevant card of the document and triggers the speech front-end engine to wait for the user 
utterance input; 

6) when a user utterance is received, the DSR client (101) performs a front-end speech 
algorithm, extracts speech features, packs the feature extraction data and then sends the feature 
extraction packets to the DSR server (140); 

7) after all the speech feature extraction data from the DSR client (101) is received, the DSR 
server (140) starts to perform speech recognition on the feature extraction data; 

8) if the speech recognition result means that the DSR client (101) needs to display another 
display card from the displayable information card document, the DSR server (140) sends an 
event notification and a relative displayable information card identifier (ID) to the DSR client 
(101) to instruct the DSR client (101) to display the corresponding card; after the DSR client 
(101) displays the identified card, the speech capture operation will be repeated from the step of 
waiting for a user utterance; 

9) if the speech recognition is unsuccessful or the utterance is not decipherable, the DSR server 
(140) sends a corresponding event notification to the DSR client (101) and the DSR client 
displays an error indication; 

10) if the speech recognition result means that the DSR client (101) needs to display a new 
document, the DSR server (140) sends a DSRML request to Web server (150), and after 
receiving the requested DSRML document, the server parsing operation will be repeated from the 
step of parsing and compiling the DSRML document. 
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[0020] In the above description, DSRML is a specialized markup language based on 
conventional Extensible Markup Language (XML) and is defined and customized for the DSR 
application system. 

[0021] It should be appreciated that the above description of the operation of a DSR application 
system is based on a particular embodiment and provided for the purpose of illustration. There 
are many variants of the DSR application system of the present invention. For example, there 
could be more DSR clients and more Web servers or DSR servers than those shown in Figure L 
Further, the networks could be different than those shown. 

[0022] As mentioned above, in a DSR application system speech the front-end engine of the 
DSR client performs feature extraction and the DSR server performs speech recognition. It is 
well known by those of ordinary skill in the art that speech recognition needs only a small part of 
the information that the speech signal carries. The representation of the speech signal used for 
recognition concentrates on the part of the signal that is related to the vocal-tract shape. So the 
data traffic generated by transmitting speech information is greatly reduced. But, all these 
operations (user utterance inputting, extracting speech features, transmitting the features to the 
DSR server, recognizing, retrieving DSRML, sending corresponding documents or events back 
to the DSR client and display feedback to the user) should be performed in a user tolerant time 
frame. 

[0023] Figure 2 is a block diagram that depicts the components of a DSR application and the 
data transmission system thereof in accordance with one embodiment of the present invention. 
The DSR application system in Figure 2 includes a DSR client (201), a DSR server (203), a Web 
server (204) and a wireless/wired gateway (202). 

[0024] As shown in Figure 2, the DSR client (201) comprises a DSR client browser (21 1) to 
allocate tasks to the components of the front-end engine (213) and the client wrapper (212), to 
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display content in the client's display screen, and to originate QoS requests. A Resource 
Reservation Setup Protocol (RSVP) module (214) supports RSVP protocol and QoS 
functionalities, such as a packet classifier, admission control, a packet scheduler, and the like. A 
front-end engine (213) is provided to reduce noise, extract speech features, and provide a speech 
feature extraction stream to the DSR client browser (21 1). A client wrapper (212) is provided to 
send connection requests, receive DSRML document contents, transmit speech feature extraction 
data, and handle events for synchronization. Additional components such as the User Datagram 
Protocol (UDP) (216), Transmission Control Protocol (TCP) (215), IP (217) modules, and the 
physical layer (218) are provided to support basic underlying network protocols. 
[0025] The DSR server (203) comprises a DSR server browser (231) to interpret DSRML 
documents, allocate the tasks to other processing engines, send display contents back to the DSR 
client after other processing engines finish their tasks, and originate QoS requests. An RSVP 
(235) module supports RSVP protocol and QoS functionalities. Other processing engines (234) 
are used to control transmissions, balance workloads, generate client content, etc., which is 
described in the related patent application referenced above. A DSR recognition engine (233) 
performs speech recognition. A server wrapper (232) receives speech feature extraction data, 
transmits and wraps DSRML content, and handles events for synchronization. Other server 
components, such as UDP (237), TCP (236), IP module (238), and the physical layer (239) are 
provided to support standard basic underlying network protocols. 

[0026] The Web server (204) comprises a web daemon (241) to process requests from the DSR 
server browser (231), produce DSRML documents in reply, and originate QoS requests. An 
RSVP module (243) supports RSVP protocol and QoS functionalities. A Hypertext Transfer 
Protocol (HTTP) wrapper (242) is provided to encapsulate and deliver HTTP application data 
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using HTTP protocol. Other Web server components, such as UDP (245), TCP (244), IP module 
(246), and the physical layer (247) are provided to support basic underlying network protocols. 
[0027] Wireless/wired gateway (202) supports wireless and wired communication between DSR 
clients and a wireless access network, such as SGSN and GGSN. 
[0028] The DSR data transmission system is composed of client (201) side components 
including the client wrapper (212), the RSVP module (214), the lower layer modules including 
UDP (216), TCP (215), IP (217), and the physical layer (218). Server (203) side components 
including the server wrapper (232), the RSVP module (235), the lower layer modules including 
UDP (237), TCP (236), DP (238) and the physical layer (239). Additional components of the 
DSR data transmission system include and the wireless/wired gateway (202). 
[0029] Figure 3 is a block diagram that depicts a DSR client wrapper (212) of the DSR data 
transmission system in accordance with one embodiment of the present invention. As shown in 
Figure 3, the client wrapper (212) is composed of a client wrapper Application Programming 
Interface (API) (301) which provides an interface between the client wrapper (212) and outside 
modules; a feature compressor (302) to compress speech feature extraction data, with which a 
vector compression algorithm could be utilized; a DSR frame constructor (303) to construct DSR 
frames; a transmission/recognition adapter (306) to adjust transmission control conditions of the 
DSR payload wrapper (304) and to control flag bits needed for recognition according to 
transmission/recognition parameters; a DSR payload wrapper (304) to construct DSR payload 
data packets, which add flag bits to the DSR packets and pass the DSR payload to corresponding 
protocol stacks according to a TCP/UDP selection; a Real-Time Transport Protocol (RTP) sender 
(305) for sending data using RTP through UDP/IP protocol stacks, which includes a buffer (not 
shown in Figure 3) to store the packets that have been sent out but not acknowledged by the DSR 
server; a DSRML client transceiver (307) for receiving DSRML data and sending an initial 
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connection request to the DSR Server, which also includes a DSRML TCP client (308) that 
implements the function of the TCP client. 

[0030] The control parameters mentioned above are used to control corresponding flexible 
options of the speech feature extraction transmission including: 

1) Frame factor: determines how many frames should be encapsulated into one DSR payload 
packet; 

2) TCP/UDP selection: indicates whether the speech features should be transmitted using TCP 
protocol or using UDP protocol; 

3) Flag bits: indicate the end of current speech input, the current sample rate, and the front-end 
type in each DSR payload packet. 

[0031] The speech features are received by client wrapper API (301) from the DSR client 
browser (211) and sent to feature compressor (302) where they are compressed using a 
conventional compression algorithm, such as vector quantization (VQ) that is well known in the 
art. The compressed speech features are then sent to the DSR frame constructor (303). The DSR 
frame constructor (303) packages the compressed speech features into a DSR frame according to 
a DSR frame format that is standardized by ETSL Then, the DSR payload wrapper (304) receives 
the compressed speech feature data in a frame format, constructs DSR payload packets 
comprising a plurality of DSR frames, and adds flag bits to the DSR packets. 
[0032] As the speech features are received from the DSR client browser (211), 
transmission/recognition parameters are also received by the client wrapper API (301) and sent to 
transmission/recognition adapter (306). Transmission/recognition adapter (306) adjusts 
transmission control conditions of the DSR payload wrapper (304) and controls flag bits needed 
for recognition according to the received transmission/recognition parameters. Therefore, the 
DSR payload wrapper (304) sends the prepared DSR packets to the RTP sender (305) or the TCP 
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module (215) according to the TCP/UDP selection in the transmission/recognition parameters. If 
the TCP/UDP selection is TCP, the DSR payload wrapper (304) sends the DSR packets to the 
TCP module (215); if the TCP/UDP selection is UDP, the DSR payload wrapper (304) sends the 
DSR packets to the RTP sender (305), and the RTP sender (305) then sends the DSR packets 
using RTP/UDP/IP protocol stacks. The RTP sender (305) has a buffer (not shown in Figure 3) 
that is used to store the DSR packets, which have been sent out but not acknowledged by the 
DSR server (203). 

[0033] GPRS performance is more optimum for large packet sizes because transmission 
overhead becomes increasingly significant as the packet size decreases, as known in the art. The 
GPRS system can handle greater input loads when transferring larger packets until the saturation 
point, at which point transfer delays increase dramatically. More input can be served with 
reasonable latency using this system. 

[0034] Therefore, in order to reduce DSR transmission overhead over GPRS, the number of 
frames included in a DSR payload packet in the DSR application is increased. Two bytes are 
also allocated in each DSR payload packet to indicate the end of current speech input, the 
number of frames included in the current packet, the current sample rate, and the front-end type. 
However, an increasing number of frames in a packet creates a risk of the failure of the speech 
recognition if packet loss or corruption occurs during the transmission. Thus, reliable delivery of 
the DSR speech feature data is of a high priority for DSR transmission over GPRS. 
[0035] Figure 4 is a block diagram that depicts a DSR server wrapper (400) of the DSR data 
transmission system in accordance with one embodiment of the present invention. As shown in 
Figure 4, the server wrapper (400) is composed of an RTP receiver (408) to receive packets using 
RTP through UDP/IP protocol stacks and extract the DSR payload from the received packets; a 
DSR payload de-wrapper (407) to separate DSR speech feature extraction data from the 
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transmission/recognition parameters; a DSR frame extractor (403) to extract DSR frames; a 
feature de-compressor (402) to de-compress speech feature extraction data; a server 
transmission/recognition adapter (404) to control frame extraction according to transmission 
parameters and to send flag bits to the server wrapper API (401) for speech recognition; a server 
wrapper API (401) to interface between the server wrapper (400) and outside modules; and a 
DSRML server transceiver (405) to send DSRML documents and receive initial connection 
requests. The DSRML server transceiver (405) also includes a DSRML TCP server (406) to 
implement the function of a TCP server. 

[0036] The processes involved in the data transmission of the DSR application system are 
illustrated by the following description with references to Figure 5 and Figure 6. 
[0037] Figure 5 is a flow chart that depicts a method for sending DSR data from the DSR client 
of a DSR application system, in accordance with one embodiment of the present invention. The 
process starts at block (505), where client wrapper API (301) receives speech features and 
transmission/recognition parameters from DSR client browser (21 1). At block (510), the received 
speech features are compressed by the feature compressor (302). Then, the compressed speech 
features are packaged into DSR frames by DSR frame constructor (303), at block (515). The 
DSR frames and flag bits in the transmission/recognition parameters are collected by DSR 
payload wrapper (304) at block (520), to form the DSR payload. Preferably, the DSR payload 
should contain the maximum number of DSR frames that the underlying transport protocol can 
support. 

[0038] Next at block (525), the DSR payload is passed to transport protocol stacks composed of 
RTP, UDP, and IP. At block (530), IP packets are sent to the DSR server (203) and each 
outgoing RTP packet is stored in a buffer. While sending the RTP packets to the DSR server 
(203), the DSR client (201) also receives corresponding Real-Time Transport Control Protocol 
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(RTCP) feedback packets concurrently, at block (535). At block (540), the stored RTP packets 
acknowledged by the received RTCP packets are freed. 

[0039] Afterwards, at block (545), a determination is made to determine if new speech features 
have been generated by the front-end engine (213) and sent to the client wrapper API (301). If 
so, then repeat the process from block (505). If no new speech features have been generated, go 
on to block (550). Another determination is made at block (550) to determine if all outgoing 
packets are acknowledged. If so, the process is ended at block (560); otherwise at block (555), 
stored packets that are not acknowledged by RTCP packets are retransmitted and then the process 
is repeated from block (535). 

[0040] Because QoS support is an option of network operators and mobile users and because 
highly reliable transmission is required for DSR applications over GPRS, TCP is used with its 
enhancement for DSR speech feature data transfer if no QoS is provided across a particular 
network. 

[0041] TCP ensures reliable end-to-end data delivery even when lower-layer services do not 
provide QoS guarantees. DSR data traffic in this application scenario is typically dominated by 
short burst transfers, which are spaced out by long idle periods while users are browsing the 
information. Short transfers and idle connections introduce much latency and degrade TCP 
performance for DSR transmission. In order to overcome these problems, in accordance with 
another embodiment of the present invention, the following steps could be taken: 
[0042] Increasing TCP initial window. Traditional TCP applies an initial window (IW) of an 
SMSS (sender maximum segment size) to transfer user data, which introduces much latency into 
DSR applications. Preferably, the TCP IW should be increased to twice the standard SMSS for 
DSR transmission, because this size reduces transfer latency significantly. It is true that with the 
augmentation of the TCP IW, packet drop rate also increases. But the increase in drop rate is less 
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than 1% if the TCP IW is set to twice the standard segment size. Thus, the increase of the TCP 
IW to twice the SMSS is worthwhile. 

[0043] Adopting no slow-start restart. The behavior of existing TCP when restarting after an idle 
period (when users are browsing obtained information) can be characterized as either no slow- 
start restart (NSSR) or slow-start restart (SSR). In the former approach, the TCP sender may send 
a large burst of back-to-back packets reusing the prior congestion window upon restarting after 
an idle connection, which risks router buffer overflow and subsequent packet loss. In the latter 
case, TCP enters slow start and initializes the current sending window to the size of the initial 
window, leading to low throughput and long latency. Taking the characteristics of DSR bit 
streams into consideration, NSSR should be selected to send DSR speech feature data preferably, 
because the gap of 10 ms between two successive frames limits the burstness of short DSR flows 
to a data rate of approximately 4600 bit/s after an idle time, thus avoiding bursty back-to-back 
packet transmission. 

[0044] Applying TCP SACK. TCP selective acknowledgment options (TCP SACK) are used as 
a means to alleviate TCP's inefficiency in handling multiple drops in a single window of data. 
Unlike the standard cumulative TCP ACKs, TCP SACK informs the sender of data that has been 
received so as to avoid retransmission of successfully delivered segments. 
[0045] Figure 6 is a flow chart that depicts a method for receiving DSR data at a DSR server of a 
DSR application system, in accordance with one embodiment of the present invention. The 
process starts at block (600), where a DSR RTP packet is received at block (605) and its 
corresponding RTCP acknowledgement packet is sent at block (620), as shown in Figure 6. At 
block (610), a determination is made to identify whether the received packet is a duplicated DSR 
RTP packet because of a fast retransmission. If it is a duplicated packet, the packet is dropped at 
block (615) and the process repeats from block (605). Otherwise, at block (625), the DSR 
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payload is de-wrapped from the DSR packet, and DSR speech feature data and 
transmission/recognition parameters are separated. Afterwards, flag bits are extracted from the 
transmission/recognition parameters at block (630), and DSR frames are extracted at block (635). 
Speech feature data is de-compressed at block (640). Then, block (645) determines whether the 
extracted flag bits indicate the end of speech. If the determination of block (645) is no, the 
process repeats from block (605). If the determination of block (645) is yes, the speech features 
and recognition parameters for recognition are sent to DSR server browser (231), and the process 
finishes at block (655). 

[0046] Accordingly, if the DSR speech feature data is sent out through TCP/IP protocol stacks, 
the receiving process should include receiving TCP packets, sending back a TCP Selective 
Acknowledgement packet to the DSR client, and the blocks (620) to (655) as shown in Figure 6 
in accordance with another embodiment of the present invention. 
[0047] In the section above, a system and method of DSR data transmission for a DSR 
application over GPRS that can transmit DSR data reliably without large latency between DSR 
server and DSR clients is described. The scope of protection of the claims set forth below is not 
intended to be limited to the particulars described in connection with the detailed description of 
the presently described embodiments. 

[0048] The present invention provides a DSR data transmission system for a DSR application 
over GPRS. The DSR application includes a plurality of DSR clients, each comprising a DSR 
client browser and a front-end engine, a DSR server comprising a DSR server browser and a 
DSR recognition-engine, and a Web server. The DSR data transmission system comprises a 
client wrapper for sending connection requests, receiving DSRML content, transmitting speech 
feature data, and handling events for synchronization; a client protocol stack for supporting 
standard underlying communication protocols; a wireless/wired gateway for supporting wireless 
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and wired communication between DSR clients and the DSR server; a server wrapper for 
receiving speech feature data, transmitting and wrapping DSRML content, and handling events 
for synchronization; and a server protocol stack for supporting standard underlying 
communication protocols. 

[0049] The present invention also provides a DSR client of a DSR application comprising a DSR 
client browser for allocating the tasks, displaying content, and originating QoS requests; a front- 
end engine for reducing noise and extracting speech features; a client protocol stack for 
supporting standard underlying communication protocols; and a DSR client wrapper for sending 
connection requests, receiving DSRML content, transmitting speech feature data, and handling 
events for synchronization. 

[0050] The present invention also provides a DSR server of a DSR application comprising: a 
DSR server browser for interpreting DSRML documents, allocating the tasks, sending display 
content back to a DSR client and originating QoS requests; a server wrapper for receiving speech 
feature data, transmitting and wrapping DSRML content, and handling events for 
synchronization; and a server protocol stack for supporting standard underlying communication 
protocols. 

[0051] Thus, a DSR data transmission system and method is described. 
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